The inventions disclosed herein relate to the field of graphics processing and, without limitation, the software manipulations that bring graphics requests from an application to the rendering or presenting hardware.
Graphics processor units (GPUs) have become important for processing data-parallel graphics tasks. Developers now recognize that non-graphics data-parallel tasks can also be handled by GPUs, taking advantage of their massive parallel capabilities. Vendors and standards organizations have created application programming interfaces (APIs) that make graphics data-parallel tasks easier to program because of the high level of developer programming interaction. However, there are also low-level APIs (libraries/frameworks, etc.) that reside closer to hardware and are generally employed by applying the output of the higher level APIs. In other words, the higher level APIs generally simply prepare program code for application to the lower level APIs.
The new landscape of graphics processing enables high levels of speed and efficiency. To access these benefits, however, custom programming and other mechanisms are generally required. For example, one standard process for a rendering pipeline begins when an application makes a graphics change resulting in a change to the current scene graph. The application uses a high-level framework/library to effect this change and, from the point of view of the application, the changes are submitted to a rendering service or rendering server. The high-level framework/library then walks the scene graph and issues drawing commands (potentially to a lower-level framework/library) to re-paint the appropriate section of the screen. Thus, the hardware is used to create the new pixels for the screen. However, often several rendering passes may be employed prior to committing content to the frame buffer. The multiple rendering passes are sometimes employed to incrementally move the data toward its displayable form. For example, effects may be sequentially applied to the same graphic element—lighting, shadows, reflections, specular illumination, etc. In addition, multiple rendering passes may be employed for creating pieces or subsets of a single frame to be composited later to form the whole frame. In any event, the use of multiple rendering passes causes latency that can be a factor depending upon the speed of the system and the complexity and rate of change of the graphics. For example, in gaming applications, the extent and complexity of graphics can be very resource demanding. Fortunately, for most gaming applications, the universe of displayable graphics is largely pre-determined. For example, a game application typically knows all the assets, state vectors, and geometries in advance. In other words, when a game loads or a game level loads, the application typically knows the substantial universe of displayable graphics that may be shown by the game. Furthermore, even considering games where the substantial universe is not known, most usually a great majority of the graphics are known in advance. This pre-determined knowledge of the displayable graphics allows gaming applications to pre-render graphic pieces and thereby avoid latency issues at runtime—i.e. at the time the graphics are demanded for the screen.
Unfortunately, much of the content used for normal display in a computing environment is not pre-known or pre-determined. For example, normal user interface actions, web pages, and even stored documents like PDFs are generally unknown to the graphics system prior to their first associated graphics request. Sometimes, the graphics are even unknown to the appropriate application (e.g. Acrobat reader does not know the contents of a document until the user opens the document). Furthermore, gaming applications are designed with the notion of heavy graphics in mind, so the applications themselves can help manage the workload and employ non-standard graphical tools and techniques.